Proceedings  -  23rd  Annual  Conference  -  lEEE/EMBS  Oct.25-28,  2001,  Istanbul,  TURKEY 


1  of  4 


THREE  DIMENSIONAL  VOLUMETRIC  MODELING  OF  THE  INTERNAL 
BRAIN  STRUCTURE  USING  MAGNETIC  RESONANCE  IMAGING  SLICES 

A.  A.  Sallam*,  M.  S.  Aboul-Soud*,  E.  H.  Mohamed^ 

^Department  of  Electrical  Engineering,  Suez  Canal  University,  Port  Said,  Egypt 


Abstraci-T\ve^  conventional  cross  sections  of  the  brain,  provided 
by  magnetic  resonance  imaging  (MRI)  scanners,  comprise  a 
sparse  dataset  of  2-D  gray-level  images,  that  is  neither  capable 
of  representing  the  3-D  nature  of  the  brain,  nor  differentiating 
its  various  component  parts  in  a  convenient  way.  The  target  of 
the  developed  work  is  to  fuse  more  information  from  the 
original  MRI  cross  sections,  which  leads  to  building  a  3-D 
computerized  color-coded  model  of  the  normal  human  brain. 
The  proposed  model  is  beneficial  in  many  areas  like  medical 
training,  radiation  treatment,  3-D  model  matching,  or  volume 
mensuration  of  brain  component  parts.  This  paper  presents  a 
revision  of  the  different  methods  for  building  3-D  brain  models, 
along  with  their  advantages  and  disadvantages.  A  proposed 
method  for  building  a  3-D  brain  model  is  then  introduced.  The 
method  consists  of  three  stages:  interpolation  of  the  original 
MRI  slices,  segmentation  of  the  different  brain  tissues,  and  3-D 
volumetric  reconstruction.  The  resultant  model  can  be 
geometrically  transformed  and  arbitrarily  dissected.  The  results 
are  shown  throughout  the  paper.  Finally,  the  conclusions  drawn 
from  this  work,  as  well  as  possible  future  extensions  of  the  work, 
are  listed. 

Keywords  -  3-D  volumetric  modeling,  brain  atlas,  image 
segmentation,  magnetic  resonance  imaging  (MRI),  slice 
interpolation 

1.  Introduction 

A  truly  3-D  feel  of  the  brain  cannot  be  obtained  from  the  2-D 
cross  sections  provided  by  medical  scanners.  Although 
radiologists  are  trained  to  reconstruct  this  picture  in  their 
minds,  this  is  of  course  a  difficult  task  for  the  untrained  eye. 
So,  it  is  of  a  great  importance  to  construct  a  comprehensive 
3-D  brain  atlas  or  model,  which  helps  in  describing  the  shape, 
configuration,  and  layout  of  brain  component  parts. 
Eurthermore,  for  3-D  visualization  of  brain  structures, 
accurate  volume  mensuration,  radiation  treatment,  or  3-D 
model  matching,  a  3-D  model  representation  of  the  normal 
human  brain  is  an  essential.  With  a  digital  brain  atlas,  the 
user  is  able  to  translate,  rotate,  scale,  and  even  dissect  the 
brain  model  in  3-D.  The  user  can  also  visualize  a  certain 
structure  from  the  point  of  view  that  best  clarifies  it.  Interest 
in  constructing  a  3-D  human  brain  model  has  been  the  focus 
of  many  researchers  [1],  [2],  [3].  Besides,  the  manufacturing 
of  new  brain-imaging  scanners  and  the  vast  development  of 
computer  systems  combine  together  to  start  a  revolution  in 
the  brain-related  field  of  research. 

In  section  II,  the  various  3-D  modeling  methods  are 
reviewed.  The  proposed  modeling  method  is  introduced  in 
section  III.  Section  IV  shows  the  different  output  results 
gained  from  this  work.  Einally,  the  conclusions  and  possible 
future  work  extensions  are  listed  in  section  V. 


II.  3-d  construction  and  modeling  methods 

There  are  several  possible  approaches  for  building  a  3-D 
brain  model.  Some  of  these  approaches  include  building  a 
mesh-based  model,  a  contour-based  model,  or  a  3-D 
volumetric  model. 

Mesh-based  models  are  concerned  in  extracting  the  iso¬ 
surface  volume  of  interest  (VOI)  using  certain  algorithms. 
These  algorithms  use  interpolative  techniques,  which  assume 
continuity  and  smooth  gradient  of  voxel  densities  [4].  The 
assumption  clearly  fails  in  the  case  of  a  color-coded  volume. 
A  natural  solution  would  be  to  convert  the  volume  into  a 
binary  volume,  based  on  the  index  value  of  the  structure  of 
interest,  then  to  isolate  the  surface  with  a  threshold  value  of 
0.5.  The  authors  in  [5]  used  this  method  for  building  a  3-D 
brain  atlas.  They  had  to  use  a  convolution-like  surface 
reconstruction  technique  to  smooth  the  original  volume.  The 
immersion  effect  is  a  drawback  of  this  method.  It  appears 
between  surfaces  of  adjacent  structures,  and  produces 
distorted  and  enlarged  surfaces.  So,  this  effect  has  to  be 
minimized.  Examples  of  other  drawbacks  of  the  mesh-based 
3-D  modeling  are  that  small  structures  are  eliminated  and  the 
resulting  model  is  jagged. 

The  second  approach  for  building  a  3-D  brain  model  is  the 
contour-based  modeling,  which  connects  the  adjacent  slices 
forming  a  wireframe  model.  Such  algorithms  can  be  quite 
simply  implemented,  and  could  be  easily  adjusted  to  the 
color-coded  images.  The  correlation  technique  is  the 
backbone  of  the  contour-based  modeling  method.  Several 
landmarks  in  each  slice  have  to  be  correlated  and  connected 
to  their  counterparts  in  the  next  one  using  matching  vectors. 
The  authors  in  [6]  adopted  this  technique  in  their  work.  The 
contour-based  3-D  modeling  incorporates  the  following 
difficulties:  anatomical  correspondences  between  slices  have 
to  be  explicitly  defined,  cross-over  connections  have  to  be 
eliminated,  and  the  missing  data  between  adjacent  slices  have 
to  be  compensated.  The  matter  gets  worse  when  a  landmark 
is  translated  from  one  slice  to  the  next  one.  Also,  branching 
objects  cause  great  difficulties. 

Three-dimensional  volumetric  modeling  is  the  third 
approach  for  building  a  3-D  brain  model.  In  this  method,  a 
pack  of  horizontal  slices  are  put  one  on  the  top  of  another. 
This  method  has  many  advantages.  Eirst,  the  generated 
models  are  best  suited  for  volume  rendering  techniques. 
Second,  the  resultant  model  can  be  arbitrarily  dissected 
without  any  difficulty.  Moreover,  the  images  of  a  patient, 
however  they  are  acquired,  can  be  easily  registered  to  the 
model.  Because  this  approach  alleviates  the  disadvantages  of 
the  other  ones,  it  is  adopted  in  this  work. 
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III.  The  proposed  modeling  method  2nd.  Segmentation  of  the  Different  Brain  Tissues 


A  volumetric  method  for  building  a  3-D  model  of  the  brain 
is  proposed.  It  is  based  on  the  following  steps  : 

1st. Interpolation  of  the  Original  Magnetic  Resonance 

Imaging  (MRI)  Slices 

This  step  assists  in  building  an  isotropic  3-D  space  and  in 
representing  the  internal  brain  structure  much  more  smoothly 
and  accurately.  Because  medical  imaging  systems  collect 
data  slice  by  slice,  the  distance  between  adjacent  image 
elements  within  a  slice  is  different  from  the  spacing  between 
adjacent  image  elements  in  two  neighboring  slices.  Eor 
visualization,  manipulation,  and  analysis  of  such  anisotropic 
data,  they  often  need  to  be  converted  into  data  of  isotropic 
discretization.  This  conversion  helps  in  visualizing  the 
imaged  property  of  the  structure  of  interest  within  any 
selected  plane  of  arbitrary  orientation  [7]. 

The  trilinear  method  of  interpolation  proved  to  be  a  good 
estimation  method  [8].  It  is  better  than  the  nearest  neighbor, 
the  averaging,  or  the  linear  interpolation  methods,  because  it 
uses  more  sample  values  to  compute  the  estimate  and  because 
the  distance  from  each  sample  to  the  interpolation  point  is 
taken  into  account.  Higher-order  interpolations  than  the 
trilinear  method  may  create  inaccurate  estimates  because  of 
the  assumed  strict  data  variation  among  points  [9].  Using  the 
kriging  interpolation  method  with  a  small  number  of 
neighborhoods  produced  interpolant  value  estimates  close  to 
the  trilinear  method,  and  got  the  closest  match  to  trilinear 
with  a  neighborhood  of  eight  voxels  [8].  But  it  should  be 
noted  that  the  trilinear  method  is  much  faster  than  the  kriging 
method. 

In  [8],  the  trilinear  interpolation  technique  was  applied  to 
each  pair  of  consecutive  slices  in  order  to  obtain  the  in- 
between  slice.  An  example  of  an  estimated  slice  along  with 
its  surrounding  slices,  from  which  the  estimated  slice  was 
obtained,  are  shown  in  Fig.  1.  At  the  end  of  this  stage,  the 
number  of  slices  has  been  doubled.  The  interpolation 
technique  was  then  applied  again,  and  finally,  an  isotropic 
dataset  was  obtained.  This  final  dataset  was  then  used  in  the 
following  segmentation  and  3-D  reconstruction  stages.  The 
validation  results  were  given  in  [8],  which  showed  the 
superiority  of  the  adopted  interpolation  technique. 


(a)  (b)  (c) 


Fig.  1 .  The  results  of  the  interpolation  stage,  (a)  The  lower  slice, 
(b)  The  in-between  estimated  slice,  (c)  The  upper  slice. 


This  step  makes  the  differentiation  among  brain  tissues 
very  easy.  Brain  matters,  as  assessed  by  MRI,  can  generally 
be  categorized  as  white  matter  (WM),  gray  matter  (GM),  or 
cerebrospinal  fluid  (CSF).  Most  brain  structures  are 
anatomically  defined  by  boundaries  of  these  tissue  classes. 
So,  a  method  to  classify  and  segment  tissues  into  these 
categories  is  an  important  step  in  quantitative  morphology  of 
the  brain.  An  accurate  classification  and  segmentation 
technique  may  facilitate  the  detection  of  various  pathological 
conditions,  3-D  visualization  of  brain  matter,  model 
registration,  radiotherapy  treatment,  and  surgical  planning 
[10],  [11]. 

It  has  been  found  that  feed-forward  artificial  neural 
network  (ANN)-based  classification  and  segmentation 
methods  generally  perform  better  than  other  algorithms  [10], 
[11].  So,  the  advantages  and  capabilities  of  ANNs  are  the 
motivations  behind  the  usage  of  an  ANN-based  segmentation 
technique  in  this  work  [12]. 

In  [13],  a  three-layer  feed- forward  ANN  is  used  to  segment 
the  original  MRI  slices  into  WM,  GM,  and  CSF.  Fig.  2  shows 
the  ANN  used.  Firstly,  a  graphical  user  interface  (GUI) 
environment  is  designed  to  enable  the  acquirement  of 
characterizing  pixels  of  each  tissue  type.  This  process  is  done 
once  and  off-line.  The  next  step  is  to  use  these  pixels  as  the 
training  vectors  of  the  ANN.  The  error  back-propagation 
training  technique  is  used  with  the  generalized  delta  rule  for 
learning.  The  presentation  phase  is  then  used  to  segment  the 
multispectral  MRI  slices.  The  presentation  phase  is  applied  to 
each  pixel  of  the  slice  in  order  to  decide  to  which  tissue  it 
belongs.  The  pixel  is  then  given  a  specific  color  representing 
this  tissue.  New  color-coded  images  are  created  as  shown  in 
Fig.  3.  The  learning  error  curve,  the  weight  and  bias  values  of 
the  ANN,  and  the  validation  results  are  found  in  [13]. 


Hidden  Layer  1  Hidden  Layer 2 


Fig.  2.  The  proposed  ANN  used  to  segment  the  brain  tissues. 
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3rd.  3-D  Volumetric  Reconstruction 

The  steps  of  this  stage  can  be  summarized  as  follows: 

1)  The  boundary  (contour)  of  each  image,  like  the  image 
shown  in  Eig.  3(c),  is  traced.  Some  points  on  the 
contour  are  marked  and  used  to  build  a  spline  that 
represents  the  contour,  as  shown  in  Eig.  4(a). 

2)  The  curve  is  then  capped  and  extruded  to  give  it  a  small 
thickness  as  shown  in  Eig.  4(b). 

3)  The  image  in  Eig.  3(c)  is  used  to  shade  the  slice  obtained 

in  step  2.  Eig.  5  shows  a  complete  and  a  cut  slices. 

4)  The  previous  steps  are  repeated  for  all  the  axial  images. 
The  slices  are  put  one  on  the  top  of  another  and  then 
linked  together  to  form  the  volumetric  model. 

5)  The  model  is  Blinn-shaded  using  two  light  sources,  one 
is  above  and  to  the  left  of  the  scene,  and  the  other  is 
below  and  to  the  right. 

6)  The  scene  is  then  rendered  using  the  scan  line  and  the  z- 
buffer  algorithms.  A  thorough  discussion  of  these 
algorithms  is  found  in  [14]. 


w 

w 

(a)  (b)  (c) 

Fig.  3.  Output  segmented  slices  representing  different 
axial  cross-sections  of  the  brain. 


Fig.  4.  (a)  The  contour  of  the  slice,  (b)  The  contour  is  given  some  depth 


Fig.  5.  (a)  Complete  shaded  slice,  (b)  Shaded  slice  with  a  cut. 


The  following  results  are  obtained: 

•  The  model  can  be  geometrically  transformed,  animated,  and 
even  dissected.  The  results  are  obtained  in  various 
projections,  which  are  helpful  in  studying  the  internal  brain 
structure  from  different  positions.  This  is  done 
interactively,  which  is  a  great  advantage  of  the 
computerized  brain  models. 

•  Several  cuts  are  made  in  the  model  to  see  the  internal 
structures  of  the  brain.  Examples  of  these  cuts  are  the 
cylindrical  and  spherical  cuts,  shown  in  Eig.  6.  Any  other 
dissection  can  be  made.  The  resulted  images  can  be  used  to 
enhance  the  understanding  of  the  brain  anatomy  or  to  help 
brain  surgeons  in  making  their  decisions. 

•  Sagittal,  coronal,  and  tilted  slices  can  be  obtained  from  the 
model  without  using  the  MRI  scanner  for  an  extra  period  of 
time  or  exposing  patients  to  extra  rays,  Eig.  7. 

The  output  results  of  the  volumetric  modeling  stage  have 
been  validated  by  medical  experts.  They  have  been  compared 
to  the  brain  anatomy  in  textbook  atlases  including  the 
Talairach-Tournoux  atlas.  The  comparisons  showed  the 
efficiency  of  the  proposed  work. 

V.  Discussion 

The  following  conclusions  can  be  drawn  from  this  work: 

•  The  volumetric  modeling  method  proved  to  be  better  than 
other  methods  for  building  3-D  brain  models. 

•  The  proposed  ANN-based  segmentation  method  yields 
better  results  and  less  error  rates  than  conventional 
methods.  This  is  due  to  the  various  advantages  and 
capabilities  afforded  by  ANNs. 


Fiss.  of  Rolando 

Body  of  lat.  ventr. 

Temp,  horn  of 
lat.  ventr. 


(a) 


(b) 

Fig.  6.  The  model  is  dissected  arbitrarily  to  show  the  internal 
brain  structure,  (a)  Cylindrical  cut.  (b)  Spherical  cut. 
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Fig.  7.  (a)  Sagittal  slice,  (b)  Coronal  slice,  (c)  Sagitto-axial  slice,  (d)  Corono-axial  slice. 


•  Applying  the  interpolation  technique,  to  get  a  densely- 
packed  dataset  of  brain  cross  sections,  yields  smoother  and 
more  accurate  results  of  the  3-D  modeling  stage. 

•  Multispectral  MRI  images  are  preferred  to  single-echo 
ones,  for  the  analysis  of  brain  datasets,  because 
multispectral  images  provide  more  information  at  the  voxel 
site  than  single-echo  images. 

There  are  several  directions  to  extend  the  proposed  work  : 

•  The  cerebral  cortex  can  be  modeled  and  included  in  the 
proposed  model. 

•  The  brain  blood  vessels  and  the  basal  ganglia  can  be 
included  in  the  proposed  model. 

•  Eunctional  images  of  the  brain  can  be  registered  to  the 
proposed  model. 

•  The  proposed  model  can  be  used  for  studying  the  aging 
factor  and  brain  pathology  development. 
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